Nested Object Watermarking: Spatial Annotation Accuracy and Approaches to Content Feature based Synchronization

Nested object annotation watermarking (NOAWM) has been introduced as a specialization within annotation watermarking, whereby hierarchical object relations are embedded in photographic images and object relations may be of visual-functional or visual-spatial nature [ViSc2006]. All previous work in the NOAWM domain has focused in this first category of relations, which denotes relationships of objects on an image that can be modeled by any functional ontology, such as Word Net, based on “is part of” (meronym/holonym) relations. Generally, the concept of NOAWM techniques requires two main properties: first, some watermarking mechanism must be provided to represent the spatial regions for each annotated object in a nonambiguous, non-overlapping way and secondly, object and class relations between the annotated objects need to be modeled and coded. In the past, four different methods have been suggested and evaluated for NOAWM, which partially follow quite different concepts. A first contribution of the final paper will be the suggestion of a theoretical framework to characterize requirements, which are specific for NOAWM, define properties from these requirements and suggest a classification of previous art. A second contribution of the paper will be a study of preciseness of the spatial annotations are preserved by four approximation schemes. In the final paper, we will therefore suggest an area-based quality measurement for this aspect, which is specifically relevant to NOAWM. Finally, the synchronization problems reported from earlier work will be addressed. One possible solution to this problem can be the approach to use content-specific features of the (part-) image as helper data for synchronization. In the final version of our paper we will discuss various theoretical approaches based on for example visual hashes, object recognition and image contouring and present experimental results for one specific approach: blob contours. Introduction Annotation watermarking, sometimes also called caption or illustration watermarking, is a specific application of image watermarking, in which supplementary information is embedded directly in the media. The goal is that additional information is intrinsically linked to media content and does not get separated from the media by non-malicious processing steps such as image cropping or compression. Nested object annotation watermarking (NOAWM) has been introduced as a specialization within annotation watermarking, whereby hierarchical object relations are embedded in photographic images and object relations may be of visual-functional or visual-spatial nature [ViSc2006]. All previous work in the NOAWM domain has focused in this first category of relations, which denotes relationships of objects on an image that can be modeled by any functional ontology, such as Word Net ([Fell1998]), based on “is part of” (meronym/holonym) relations. Generally, the concept of NOAWM techniques requires two main properties: first, some watermarking mechanism must be provided to represent the spatial regions for each annotated object in a nonambiguous, non-overlapping way and secondly, object and class relations between the annotated objects need to be modeled and coded. In the past, four different methods have been suggested and evaluated for NOAWM, which partially follow quite different concepts. A first contribution of the final paper will be the suggestion of a theoretical framework to characterize requirements, which are specific for NOAWM, define properties from these requirements and suggest a classification of previous art. Basis for the later will be a review of the methods [ViSD2006], [ViDi2007] and [ViSc2007] with respect to their concept for spatial and hierarchical annotations and their embedding domains, as shown in a generalized way in Table 1. NOAWM method Spatial annotations Hierarchical annotations Domain for spatial information Domain for object hierarchy information [ViSD2006] rectangular areas object hierarchy coding Spatial domain (block-luminance) Spatial domain (blockluminance) [ViDi2007] rectangular areas object hierarchy coding Spatial domain (WPC in Blue Channel) Spatial domain (blockluminance) [ViDi2007] rectangular areas intrinsic signal energy relations Frequency domain (phase modulation) Frequency domain (magnitude modulation) [ViSc2007] block-based polygon approximation intrinsic signal energy relations Frequency domain (phase modulation) frequency domain (magnitude modulation) Table 1: Overview of NOAWM methods suggested in the past. From a detailed review of the four earlier methods, we will derive selected specific challenges in this domain, which will be further addressed: Degradation of spatial annotation accuracy due to block-based approximation: [ViSc2007] suggests a method for an approximation of the user-defined polygonal annotation shape, which apparently leads to a perceptually more precise representation of the object regions than the previous rectangular annotation approaches [ViSc2006] and [ViDi2007]. This suggested approximation is based on block-based presence watermarks, whereby the embedding area is defined by those blocks whose count of vertices inside the original polygon amounts a specific threshold in the range of {1, ..., 4}. The degradation problem is shown on the exemplary images in Figure 1, where the original, polygonal annotation is shown in the left images, the resulting approximations by rectangles and the block-based approach from [ViSc2007]. Figure 1: Original Spatial Annotations by a user (left) and their approximation by rectangular areas (middle) and block-based approximation (right) While the impact of block-based spatial approximation with respect to (overall image) transparency & robustness has already been studied in the original paper, measurements, as how precise the spatial annotations are preserved by each of the approximation schemes, have not been performed to date. In the final paper we will therefore suggest an area-based quality measurement for this aspect, which is specifically relevant to NOAWM. We will further present experimental results from a database of 15 images with manually performed object annotations, each of which contains at least 9 and at most 11 hierarchically nested objects with an overall depth between two and three levels, comparing the gain in overall image transparency, as well as geometrical shape approximation. The embedding takes advantage of the polygonal support to handle children watermarks located in the spatial area of its parent. To avoid overwriting of one watermark with another, the larger one is always embedded around the smaller one as seen from the example in Figure 2. w1,1 w1,2 w1,3 w1,4 w1,5 w1,6 w1,7 w1,8 w1,9 w1,10 w2,1 w2,2 w2,3 w1,11w1,12 w1,13w1,14 w2,4 w2,5 w2,6 w2,7 w1,15w1,16w1,17 w1,18w1,19 w2,8 w2,9 w2,10w1,20w1,21w1,22w1,23 w1,24w1,25w1,26w1,27w1,28w1,29w1,30w1,31